Audiovisual speech integration: modulatory factors and the link to sound symbolism

In this talk, I will review some of the latest findings from the burgeoning literature on the audiovisualintegration of speech stimuli. I will focus on those factors that have been demonstrated to influence thisform of multisensory integration (such as temporal coincidence, speaker/gender matching, andattention; Vatakis & Spence, 2007, 2010). I will also look at a few of the other factors that appear notto matter as much as some researchers have argued that they should, such as spatial coincidence(except under a particular subset of circumstances; Spence 2013). I will also look at the emergingliterature that has documented the widespread existence of audiovisual crossmodal correspondences,and consider their putative link to the literature on embodied cognition (Spence & Deroy, 2013). References[1] Spence, C. (2013). Just how important is spatial coincidence to multisensory integration? Evaluating thespatial rule. Annals of the New York Academy of Sciences.[2] Spence, C., & Deroy, O. (2012). Hearing mouth shapes: Sound symbolism and the reverse McGurk effect. i-Perception, 3, 550-5526.[3] Vatakis, A., & Spence, C. (2007). Crossmodal binding: Evaluating the “unity assumption” using audiovisualspeech stimuli. Perception & Psychophysics, 69, 744-756.[4] Vatakis, A., & Spence, C. (2010). Audiovisual temporal integration for complex speech, object-action,animal call, and musical stimuli. In M. J. Naumer & J. Kaiser (Eds.), Multisensory object perception in theprimate brain (pp. 95-121). New York: Springer. http://psyweb.psy.ox.ac.uk/xmodal/members.htm International Conference on Auditory-Visual Speech ProcessingAVSP2013